Join Size Estimation Over Data Streams Using Cosine Series

In many applications, data takes the form of a continuous stream rather than a persistent data set. Data stream processing is generally an on-line, one-pass process and is required to be time and space efficient too. In this paper, we develop a framework for estimating join size over the data streams based on the discrete cosine transform (DCT). The DCT generally can provide concise and accurate approximations to data distributions and its coefficients can be updated easily in the presence of insertions and deletions. These features make the DCT suitable for dynamic data stream environments. We have performed analyses and conducted experiments to investigate the applicability of the cosine transform to data streams. The experimental results show that given the same amount of storage space, our method yields more accurate estimates most of the time than the sketch-based methods, which have become the main methods for approximate query processing over data streams. The experimental results have also confirmed that the cosine series can be updated quickly to cope with the rapid flow of data streams. Keyword: Data Stream, Cosine Series, Query Estimation